Papers with paraphrase identification
NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measures (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) enables several similarity measures to estimate the probability of translations. |
| Approach: | They propose to rank the similarity of short text segments using translation-based similarity measures . they use the NMTScore library to analyze translation-related similarity . |
| Outcome: | The proposed measures show a relatively high correlation to human judgments when used for reference-based evaluation of data-to-text generation in 2 tasks and 17 languages. |
Pointwise Paraphrase Appraisal is Potentially Problematic (2020.acl-srw)
Copied to clipboard
| Challenge: | prevailing methods for paraphrase identification models are binary classification problems . current methods do not provide consistent and robust performance on unseen samples and real world problems. |
| Approach: | They propose to use binary classification to evaluate paraphrase identification models . they propose to improve methods for fine-tuning BERT models by pairing two sentences as one sequence . |
| Outcome: | The proposed methods may fail on simple tasks like identifying pairs with two identical sentences. |
Character-Based Neural Networks for Sentence Pair Modeling (N18-2)
Copied to clipboard
| Challenge: | Sentence pair modeling is critical for many NLP tasks, such as paraphrase identification and semantic textual similarity. |
| Approach: | They propose to use subwords to represent sentences without pretrained word embeddings . they find that subword models can achieve new state-of-the-art results without pretraining . |
| Outcome: | The proposed models can achieve state-of-the-art results on two social media datasets and competitive results on news data for paraphrase identification. |
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work shows that pre-trained language models perform poorly on challenging datasets where spurious correlations do not hold. |
| Approach: | They propose to use multi-task learning to improve generalization from minority examples . they propose to combine MTL with auxiliary tasks to improve performance . |
| Outcome: | The proposed model generalizes from minority examples without hurting in-distribution performance. |
Dice Loss for Data-imbalanced NLP Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension . |
| Approach: | They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks. |
| Outcome: | The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks. |
Cross-functional Analysis of Generalization in Behavioral Learning (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing evaluation paradigms for behavioral learning use correlations in training data, but they ignore important model properties such as fairness. |
| Approach: | They propose an analysis method for evaluating behavioral learning considering generalization across dimensions of different granularity levels. |
| Outcome: | The proposed method optimizes behavior-specific loss functions and evaluates models on several partitions of the behavioral test suite controlled to leave out specific phenomena. |
Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Task-adaptive pre-training (TAPT) and Self-training can be complementary with simple TFS protocol. |
| Approach: | They propose to use task-adaptive pre-training and self-training to combine TAPT and ST with a simple TFS protocol to achieve strong combined gains across six datasets. |
| Outcome: | The proposed method can achieve strong combined gains across six datasets covering sentiment classification, paraphrase identification, natural language inference, named entity recognition and dialogue slot classification. |
How much pretraining data do language models need to learn syntax? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining methods are convenient, but expensive in terms of time and resources. |
| Approach: | They investigate the impact of pretraining data size on the syntactic capabilities of RoBERTa by using syntaktic structural probes to determine whether models pretrained on more data encode a higher amount of syntastic information. |
| Outcome: | The proposed models perform better on part-of-speech tagging, dependency parsing and paraphrase identification. |
Learning Context-Sensitive Convolutional Filters for Text Processing (D18-1)
Copied to clipboard
| Challenge: | Convolutional neural networks (CNNs) are a popular building block for natural language processing . despite their success, most existing CNN models share the same learned set of filters for all input sentences. |
| Approach: | They propose to use a meta network to learn context-sensitive convolutional filters for text processing by using a bidirectional filter generation mechanism. |
| Outcome: | The proposed framework outperforms standard and attention-based CNN models on four different tasks. |
Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching (D19-1)
Copied to clipboard
| Challenge: | Sentence matching is a key issue in natural language inference and paraphrase identification. |
| Approach: | They propose a semantics-oriented attention and deep fusion network (OSOA-DFN) that is oriented to the original semantic representation of another sentence and propagates attention information at each matching layer. |
| Outcome: | The proposed model can model sentence matching more precisely on three sentence matching benchmark datasets. |
Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to explain neural network models are computationally inefficient for text inputs. |
| Approach: | They propose a method to implicitly detect word correlations by grouping correlated words from input text pairs together and measuring their contribution to corresponding NLP tasks. |
| Outcome: | The proposed method is evaluated with two different model architectures across four datasets. |
Evaluating Multilingual Sentence Representation Models in a Real Case Scenario (2022.lrec-1)
Copied to clipboard
| Challenge: | a recent study has shown that the infamous Protocols are actually plagiarized . a convoluted task with no standard benchmarks for paraphrase detection and sentence similarity is a problem . |
| Approach: | They evaluate sentence representation models on the paraphrase detection task . they use a forged text from the so-called "Protocols of the Elders of Zion" scholars have demonstrated that the first text plagiarizes from the second . |
| Outcome: | The proposed model is based on the forged “Protocols of the Elders of Zion” . the model is similar to the standard model but has some problems . |
Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering (C18-1)
Copied to clipboard
| Challenge: | Sentence pair modeling is a fundamental technique underlying many NLP tasks. |
| Approach: | They analyze several neural network designs for sentence pair modeling and compare their performance extensively across eight datasets. |
| Outcome: | The proposed models perform well across eight datasets including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks. |
Adversarial Semantic Collisions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate semantic collisions for NLP tasks are vulnerable to adversarial examples. |
| Approach: | They propose gradient-based approaches for generating semantic collisions given white-box access to a model and deploy them against several NLP tasks. |
| Outcome: | The proposed approaches evade perplexity-based filtering and discuss other potential mitigations. |
Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarks (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have highlighted the vulnerability of deep learning models to adversarial attacks. |
| Approach: | They propose a benchmark for textual adversarial defence that evaluates state-of-the-art defence mechanisms across diverse datasets, models, and tasks. |
| Outcome: | The proposed benchmark incorporates a wide range of datasets and evaluates state-of-the-art defence mechanisms. |
Simple and Effective Text Matching with Richer Alignment Features (P19-1)
Copied to clipboard
| Challenge: | Existing models only use a single inter-sequence alignment layer to make full use of this process. |
| Approach: | They propose to keep three key features available for inter-sequence alignment . they conduct experiments on four well-studied benchmark datasets . |
| Outcome: | The proposed model is able to perform on four well-studied datasets with fewer parameters and the inference speed is at least 6 times faster than similar models. |
A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to text matching consider each sequence separately . a proposed model uses both sequences to generate a given relationship with a source sequence . |
| Approach: | They propose a latent variable model for predicting the relationship between a pair of text sequences by generating a sequence that has a given relationship with a source sequence. |
| Outcome: | The proposed model achieves state-of-the-art on natural language inference and paraphrase identification. |
Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences (D18-1)
Copied to clipboard
| Challenge: | a long standing problem in NLP research is learning a matching function between two text sequences . a deep architecture for this task is proposed by a team of researchers . |
| Approach: | They propose a new deep matching model using stacked recurrent encoders to learn affinity weights . they conduct extensive experiments on six well-studied text sequence matching datasets a plethora of applications are possible . |
| Outcome: | The proposed model improves performance on six well-studied text sequence matching datasets. |
Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling (D19-1)
Copied to clipboard
| Challenge: | Existing techniques for relevance and semantic matching cannot be easily adapted to the other. |
| Approach: | They propose a model that incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness. |
| Outcome: | The proposed model incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness. |
Transfer Fine-Tuning: A BERT Case Study (D19-1)
Copied to clipboard
| Challenge: | Recent advances in sentence representation learning have improved the performance of BERT models, but the computational power required is an obstacle preventing practical applications from adopting the technology. |
| Approach: | They propose to inject phrasal paraphrase relations into BERT to generate suitable representations for semantic equivalence assessment instead of increasing model size. |
| Outcome: | The proposed model improves a smaller model while maintaining the model size. |
PerPaDa: A Persian Paraphrase Dataset based on Implicit Crowdsourcing Data Collection (2022.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we present a dataset that is collected from users’ input in a plagiarism detection system. |
| Approach: | They propose to use a Persian paraphrase dataset that is collected from users’ input in a plagiarism detection system to improve the quality of the data. |
| Outcome: | The proposed dataset contains 2446 instances of paraphrasing. |
Improving Paraphrase Detection with the Adversarial Paraphrasing Task (2021.acl-long)
Copied to clipboard
| Challenge: | a new adversarial method of paraphrase identification is being used to identify paraphrases based on word overlap and syntax . authors propose a dataset that generates semantically equivalent but lexically and syntactically disparate paraphrase pairs . |
| Approach: | They propose an adversarial method for paraphrase identification that uses word overlap and syntax to identify paraphrases. |
| Outcome: | The proposed method improves paraphrase detection accuracy and speed of generation of datasets. |
Unsupervised Paraphrasing without Translation (P19-1)
Copied to clipboard
| Challenge: | Recent work on automatic paraphrasing focuses on methods leveraging machine translation as an intermediate step. |
| Approach: | They propose to learn paraphrasing models only from a monolingual corpus . they propose a residual variant of vector-quantized variational auto-encoder . |
| Outcome: | The proposed model outperforms supervised and unsupervised translation methods in paraphrase identification and training set augmentation. |
PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Paraphrase identification requires specialized domain knowledge to perform . state-of-the-art neural models and non-expert human annotators have poor performance on PARADE . |
| Approach: | They propose a benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge. |
| Outcome: | The proposed dataset shows state-of-the-art models and non-expert human annotators have poor performance on PARADE. |
Improving Large-scale Paraphrase Acquisition and Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing Twitter-based paraphrase datasets lack quality definitions for identification and generation tasks. |
| Approach: | They propose to use two separate definitions of paraphrase for identification and generation tasks in existing Twitter-based paraphrase datasets. |
| Outcome: | The proposed model achieves state-of-the-art performance of 84.2 F1 for automatic paraphrase identification compared to other models fine-tuned on other corpora such as Quora, MSCOCO, and ParaNMT. |
Improving word mover’s distance by leveraging self-attention matrix (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Word mover's distance (WMD) computes semantic similarity via the optimal alignment between word embeddings, but does not consider word order, making it challenging to distinguish sentences with significant overlaps of similar words. |
| Approach: | They propose to incorporate the sentence structure represented by BERT’s self-attention matrix (SAM) to improve WMD and its variants in paraphrase identification. |
| Outcome: | The proposed method improves WMD and its variants in paraphrase identification with near-equivalent performance in semantic textual similarity. |
Interventional Training for Out-Of-Distribution Natural Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for NLU training use only known and single confounders, but in many NLU tasks the confounder can be unknown and multifactorial. |
| Approach: | They propose a method that performs multi-granular intervention with identified multifactorial confounders by using a bottom-up automatic intervention method. |
| Outcome: | The proposed method performs multi-granular intervention with identified multifactorial confounders on three NLU tasks, namely, natural language inference, fact verification and paraphrase identification. |
Macedon: Minimizing Representation Coding Rate Reduction for Cross-Lingual Natural Language Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to learn cross-lingual models require limited data to perform cross-linguistic tasks. |
| Approach: | They propose a method to remove language-associated information via minimizing representation coding rate reduction. |
| Outcome: | The proposed model outperforms state-of-the-art models on cross-lingual tasks. |